Control in moving interfaces and deep learning

Tesis Doctoral inédita leída en la Universidad Autónoma de Madrid, Facultad de Ciencias, Departamento de Matemáticas. Fecha de Lectura: 14-05-2021This thesis has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No.765579-ConFlex

Controllability of one-dimensional viscous free boundary flows

In this work, we address the local controllability of a one-dimensional free boundary problem for a fluid governed by the viscous Burgers equation. The free boundary manifests itself as one moving end of the interval, and its evolution is given by the value of the fluid velocity at this endpoint. We prove that, by means of a control actuating along the fixed boundary, we may steer the fluid to constant velocity in addition to prescribing the free boundary's position, provided the initial velocities and interface positions are close enough

Sparse approximation in learning via neural ODEs

We consider the neural ODE and optimal control perspective of supervised learning with $L^1(0,T;\mathbb{R}^{d_u})$ control penalties, where rather than only minimizing a final cost for the state, we integrate this cost over the entire time horizon. Under natural homogeneity assumptions on the nonlinear dynamics, we prove that any optimal control (for this cost) is sparse, in the sense that it vanishes beyond some positive stopping time. We also provide a polynomial stability estimate for the running cost of the state with respect to the time horizon. This can be seen as a \emph{turnpike property} result, for nonsmooth functionals and dynamics, and without any smallness assumptions on the data, both of which are new in the literature. In practical terms, the temporal sparsity and stability results could then be used to discard unnecessary layers in the corresponding residual neural network (ResNet), without removing relevant information.Comment: 24 pages, 5 figure

Controllability of one-dimensional viscous free boundary flows

In this work, we address the local controllability of a one-dimensional free boundary problem for a fluid governed by the viscous Burgers equation. The free boundary manifests itself as one moving end of the interval, and its evolution is given by the value of the fluid velocity at this endpoint. We prove that, by means of a control actuating along the fixed boundary, we may steer the fluid to constant velocity in addition to prescribing the free boundary's position, provided the initial velocities and interface positions are close enoughThis project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No.765579-ConFlex. E.Z. has received funding from the Alexander von Humboldt-Professorship program, the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement NO. 694126-DyCon), the Transregio 154 Project “Mathematical Modelling, Simulation and Optimization Using the Example of Gas Networks” of the German DFG, grant MTM2017-92996-C2-1-R COSNET of MINECO (Spain) and by the Air Force Office of Scientific Research (AFOSR) under Award NO. FA9550-18-1-024

Control of the linearized Stefan problem in a periodic box

In this paper we consider the linearized one-phase Stefan problem with surface tension, set in the strip $\mathbb{T}\times(-1,1)$, thus with periodic boundary conditions respect to the horizontal direction $x_1\in\mathbb{T}$. When the support of the control is not localized in $x_1$, namely, is of the form $\omega=\mathbb{T}\times(c,d)$, we prove that the system is null-controllable in any positive time. We rely on a Fourier decomposition with respect to $x_1$, and controllability results which are uniform with respect to the Fourier frequency parameter for the resulting family of one-dimensional systems. The latter results are also novel, as we compute the full spectrum of the underlying operator for the non-zero Fourier modes. The zeroth mode system, on the other hand, is seen as a controllability problem for the linear heat equation with a finite-dimensional constraint. We extend the controllability result to the setting of controls with a support localized in a box: $\omega=(a,b)\times(c,d)$, through an argument inspired by the method of Lebeau and Robbiano, under the assumption that the initial data are of zero mean. Numerical experiments motivate several challenging open problems, foraying even beyond the specific setting we deal with herein

The emergence of clusters in self-attention dynamics

Viewing Transformers as interacting particle systems, we describe the geometry of learned representations when the weights are not time dependent. We show that particles, representing tokens, tend to cluster toward particular limiting objects as time tends to infinity. Cluster locations are determined by the initial tokens, confirming context-awareness of representations learned by Transformers. Using techniques from dynamical systems and partial differential equations, we show that the type of limiting object that emerges depends on the spectrum of the value matrix. Additionally, in the one-dimensional case we prove that the self-attention matrix converges to a low-rank Boolean matrix. The combination of these results mathematically confirms the empirical observation made by Vaswani et al. [VSP'17] that leaders appear in a sequence of tokens when processed by Transformers